Systems and methods for data storage and processing

ABSTRACT

An imaging device described herein for capturing image data comprises an optical sensor responsive to electromagnetic radiation, wherein the sensor exhibits a photo-memory effect causing the sensor to act as a short-term memory device; an exposure control device configured to adopt a first mode, wherein electromagnetic radiation reaches the sensor and a second mode, wherein electromagnetic radiation is restricted from reaching the sensor; a processor configured to receive sensor data from the sensor, and to cause the exposure control device to selectively adopt the first mode and the second mode; and a long term memory device accessible to the processor, wherein the processor is configured to read executable instructions from the long term memory and to write sensor data to the long term memory. Data stored in the sensor acting as a short-term memory device is erasable by exposing the photo-sensor to electromagnetic radiation.

CROSS-REFERENCE TO RELATED APPLICATIONS

This Application is a continuation of International Patent ApplicationNo. PCT/AU2021/051331, filed on Nov. 10, 2021, which claims priority toAU Application No. 2020904109, filed on Nov. 10, 2020, the entirecontents of each of which are hereby incorporated by reference.

BACKGROUND Field

Embodiments generally relate to systems and methods for data storage andprocessing. In particular, embodiments relate to systems and methods foroptoelectronic data storage and processing.

Description of Related Art

Processes for capturing, storing and processing images are used in anumber of industries and fields. In many cases, it is advantageous forthe image capture, storage and processing to occur quickly. However,known processes often require transferring images between an imagecapture device such as a camera to a separate processing device such asa computer before image processing can occur. This introduces delays andinefficiencies which can negatively impact many computer visionapplications which require rapid and close to real-time processing,including neuro-robotics, human-machine interaction technologies andbionic systems.

It is desired to address or ameliorate one or more shortcomings ordisadvantages associated with prior systems and methods for data storageand processing, or to at least provide a useful alternative thereto.

Throughout this specification the word “comprise”, or variations such as“comprises” or “comprising”, will be understood to imply the inclusionof a stated element, integer or step, or group of elements, integers orsteps, but not the exclusion of any other element, integer or step, orgroup of elements, integers or steps.

Any discussion of documents, acts, materials, devices, articles or thelike which has been included in the present specification is not to betaken as an admission that any or all of these matters form part of theprior art base or were common general knowledge in the field relevant tothe present disclosure as it existed before the priority date of each ofthe appended claims.

SUMMARY

Some embodiments relate to an imaging device for capturing image data,the imaging device comprising: an optical sensor responsive toelectromagnetic radiation, wherein the sensor exhibits a photo-memoryeffect causing the sensor to act as a short-term memory device; anexposure control device configured to adopt a first mode and a secondmode, wherein in the first mode the exposure control device causeselectromagnetic radiation to reach the sensor and in the second mode theexposure control device restricts electromagnetic radiation fromreaching the sensor; a processor configured to receive sensor data fromthe sensor, and to cause the exposure control device to selectivelyadopt the first mode and the second mode; and a long term memory deviceaccessible to the processor, wherein the processor is configured to readexecutable instructions from the long term memory and to write sensordata to the long term memory; wherein data stored in the sensor actingas a short-term memory device is erasable by exposing the photo-sensorto electromagnetic radiation.

In some embodiments, the optical sensor is responsive to one or more ofultraviolet light, visible light, and infrared light.

According to some embodiments, the optical sensor comprises atwo-dimensional semiconductor. In some embodiments, the two-dimensionalsemiconductor comprises at least one of an elemental material,transitional metal oxide, or transition metal chalcogenide. In someembodiments, the two-dimensional semiconductor comprises blackphosphorous.

According to some embodiments, the optical sensor comprises an array ofoptical sensor devices.

In some embodiments, the sensor device is configured to exhibit areference conductance in the absence of electromagnetic radiation, andto exhibit a wavelength dependent increase and/or decrease inconductance when exposed to electromagnetic radiation. In someembodiments, the conductance of the sensor device is configured toreturn to the reference conductance not less than 1 second afterexposure to the electromagnetic radiation has ceased. In someembodiments, the conductance of the sensor device is configured toreturn to the reference conductance not less than 1 minute afterexposure to the electromagnetic radiation has ceased.

According to some embodiments, the conductance of the sensor deviceincreases when the sensor device is exposed to electromagnetic radiationof a first wavelength, and decreases when the sensor device is exposedto electromagnetic radiation of a second wavelength. According to someembodiments, the first wavelength is between 250 and 315 nm, and thesecond wavelength is between 315 and 400 nm.

In some embodiments, the processor is configured to read data stored onthe sensor device by applying a voltage to the sensor device.

According to some embodiments, the exposure control device comprises atleast one of a shutter configured to block electromagnetic radiation ora filter configured to filter out at least some wavelengths ofelectromagnetic radiation.

According to some embodiments, the exposure control device comprises atleast one source of electromagnetic radiation.

Some embodiments relate to a method of capturing image data, the methodcomprising: operating an exposure control device to adopt a first mode,wherein in the first mode the exposure control device causes an opticalsensor responsive to electromagnetic radiation to be exposed to firstelectromagnetic radiation, wherein the sensor exhibits a photo-memoryeffect causing the sensor to act as a short-term memory device; upondetermining that a threshold period of time has elapsed, operating theexposure control device to adopt a second mode, wherein in the secondmode the exposure control device causes the sensor to be blocked fromelectromagnetic radiation; reading data captured by the sensor based onthe exposure; and recording the read data to a long term memory device.

In some embodiments, the method is performed by the device of some otherdescribed embodiments.

Some embodiments further comprise erasing the read data from the sensoracting as a short term memory device by operating the exposure controldevice to adopt a third mode, wherein in the third mode the exposurecontrol device causes the sensor to be exposed to second electromagneticradiation.

In some embodiments, the exposure control device comprises a shutter orfilter, wherein adopting a first mode comprises opening the shutter orfilter, and wherein adopting the second mode comprises closing theshutter or filter.

In some embodiments, the exposure control device comprises a source ofelectromagnetic radiation, wherein adopting a first mode comprisesswitching on the source of electromagnetic radiation, and whereinadopting the second mode comprises switching off the source ofelectromagnetic radiation.

According to some embodiments, reading data captured by the sensorcomprises applying a voltage to the sensor.

Some embodiments relate to a method of training a neural network, themethod comprising: receiving at least one training image and a labelrelated to the at least one training image; capturing image data byperforming the method of some other embodiments, wherein the firstelectromagnetic radiation is generated based on the at least onetraining image; and determining an output based on the read data and aweight matrix; comparing the read data with the label; and modifying theweight matrix based on the comparison.

Some embodiments further comprise repeating the method with the same atleast one training image.

Some embodiments further comprise repeating the method with a furthertraining image different to the at least one training image.

DESCRIPTION OF THE DRAWINGS

Embodiments are described in further detail below, by way of example andwith reference to the accompanying drawings, in which:

FIG. 1 shows a block diagram of a system for data storage and processingaccording to some embodiments.

FIG. 2 shows a detailed view of an optical sensor device of FIG. 1 .

FIGS. 3A and 3B show graphs illustrating write and erase functions ofthe optical sensor device of FIG. 2 .

FIGS. 4A, 4B and 4C illustrating write, erase and read functions of theoptical sensor device of FIG. 2 .

FIGS. 5A, 5B and 5C show graphs illustrating the optical sensor deviceof FIG. 2 acting as multi-bit memory.

FIGS. 6A and 6B show transfer function of the optical sensor device ofFIG. 2 .

FIGS. 7A and 7B show the optical sensor device of FIG. 2 acting as shortand long term memory.

FIG. 8 shows a method of training the optical sensor device of FIG. 2for handwriting recognition.

FIGS. 9A and 9B show the results of optical sensor device of FIG. 2after the method of FIG. 8 .

FIG. 10 shows a method of training the optical sensor device of FIG. 2for facial recognition.

FIG. 11 shows the results of optical sensor device of FIG. 2 after themethod of FIG. 10 is performed with varying parameters.

FIG. 12 shows a flowchart illustrating a method of training the systemof FIG. 1 to perform image recognition according to some embodiments.

FIG. 13 shows a flowchart illustrating a method of capturing image datausing the imaging device of FIG. 1 .

DETAILED DESCRIPTION

Embodiments generally relate to systems and methods for data storage andprocessing. In particular, embodiments relate to systems and methods foroptoelectronic data storage and processing.

Conventionally in image capture devices such as cameras, a photo-sensoris used to generate electrical signals based on exposure to light. Thephoto-sensor cannot store the captured image data, as conventionalphoto-sensors exhibit fast recovery of photo-response after light isswitched off. To store the data, the electrical signals are read by anauxiliary circuit and data is stored to a separate electronic memorydevice in the form of “1”s and “0”s which are associated with theconductance states of the electronic memory. The two data states can bemanipulated by way of write and erase procedures, where a writeprocedure causes a memory location to take on a “1” state and an eraseprocedure causes the device to take on a “0” state. These write anderase procedures are performed electronically by changing or switchingthe conductance of the memory locations in the memory device. However,the use of electronically programmable memory devices can result inprocessing delays. They generate large amounts of redundant data whichrequires significant computing to extract meaningful information, andalso require a large amount of data storage. These requirements resultin increased data latency and high power consumption. Furthermore,electrical signals induce electronic noise in circuits, which can affectthe performance and accuracy of data processing and neuromorphicprocesses.

In contrast, described embodiments relate to systems and methods fordata storage and processing that make use of a sensor exhibiting a fullylight controlled memory effect, that allows for data to be both writtenand erased optically by way of exposure to electromagnetic radiationsuch as light. This sensor can therefore be controlled without relyingon the application of alternating polarity electric signals. This ismade possible by using a sensor element exhibiting a persistentphotocurrent in the absence of exposure to electromagnetic radiationsuch as light, having slower recovery of photo-response thanconventional photo-sensors. Furthermore, these embodiments allow forin-pixel data storage and processing, meaning processing can occurwithin the optical sensor capturing the data, without requiring the datato be transferred to an electronic memory device. The sensor cantherefore act as both a light sensor and a memory device. This may allowfor faster data processing speeds due to high bandwidth, low parasiticcrosstalk and ultralow power consumption as well as high scalability.This may allow for processes such as neuromorphic pre-processing andimage recognition, visual memory, wavelength selective multi-bitprogramming, and in-pixel image pre-processing, for example. Theseprocesses may be applied to industries including automation, securityand surveillance, brain-machine interface and bionics, among others.

FIG. 1 shows a block diagram of a system 100 for capturing, storing andprocessing image data according to some embodiments.

System 100 comprises an imaging device, being image capture device 110.Image capture device 110 may comprise a camera or scanner device in someembodiments. According to some embodiments, image capture device 110 maybe configured to act as a neuromorphic imaging device that may be usedwithin photonic neuromorphic circuits and artificial visual systems, forexample.

Device 110 includes an optical sensor device 120, which may beconfigured to also act as a short term memory device, as described infurther detail below. Optical sensor device 120 may comprise at leastone semiconductor in some embodiments. According to some embodiments,optical sensor device 120 may comprise an array of semiconductors actingas pixels for capturing an array of image data.

Optical sensor device 120 may be sensitive to electromagnetic radiation.According to some embodiments, optical sensor device 120 may besensitive to particular wavelengths of electromagnetic radiation. Forexample, in some embodiments, optical sensor device 120 may be sensitiveto light. In some embodiments, optical sensor device 120 may besensitive to visible light. Optical sensor device 120 may be sensitiveto light with wavelengths between 300 nm and 800 nm, for example. Insome embodiments, optical sensor device 120 may be sensitive to one ormore of ultraviolet light, visible light, and/or infrared light. Opticalsensor device 120 may be sensitive to light with wavelengths between 280nm and 1300 nm, for example.

In some embodiments, optical sensor 120 may comprise at least onetwo-dimensional semiconductor or semiconductor array. In someembodiments, optical sensor 120 may comprise at least onetwo-dimensional semiconductor or semiconductor array comprising anelemental material, such as black phosphorous, for example. In someembodiments, optical sensor 120 may comprise at least onetwo-dimensional semiconductor or semiconductor array comprising atransitional metal oxide, such as molybdenum trioxide, for example. Insome embodiments, optical sensor 120 may comprise at least onetwo-dimensional semiconductor or semiconductor array comprising atransition metal chalcogenide, such as tin sulphide (SnS) or molybdenumdisulphide (MoS2), for example.

According to some embodiments, the semiconductor may be thetwo-dimensional black phosphorous semiconductor described inInternational Application No. PCT/AU2019/050662, the entirety of whichis herein incorporated by reference.

According to some embodiments, the optical sensor device 120 may exhibita reference conductance when not exposed to electromagnetic radiationsuch as light, and may exhibit wavelength dependent increase anddecrease in conductance when exposed to particular wavelengths ofelectromagnetic radiation. The conductance exhibited by the opticalsensor device 120 may increase when exposed to electromagnetic radiationof a first wavelength, and decrease when exposed to electromagneticradiation of a second wavelength. For example, according to someembodiments, the conductance exhibited by optical sensor device 120 mayincrease when exposed to light with a wavelength in the range of 250-315nm, and may decrease when exposed to light with a wavelength in therange of 315-400 nm. Write and erase procedures can therefore beperformed on optical sensor device 120 by exposing optical sensor device120 to particular electromagnetic radiation wavelengths, and withoutusing electronic write or erase procedures.

According to some embodiments, after exposure to the electromagneticradiation has stopped, optical sensor device 120 may exhibit opticalmemory, with conductance persisting in the absence of electromagneticradiation, allowing optical sensor device 120 to act as a short-termmemory device. According to some embodiments, optical sensor device 120may exhibit a photocurrent for over 1 second after exposure toelectromagnetic radiation has stopped. According to some embodiments,optical sensor device 120 may exhibit a photocurrent for over 1 minuteafter exposure to electromagnetic radiation has stopped. According tosome embodiments, optical sensor device 120 may exhibit a photocurrentfor over 10 minutes after exposure to electromagnetic radiation hasstopped. According to some embodiments, optical sensor device 120 mayexhibit a photocurrent for over 1 hour after exposure to electromagneticradiation has stopped. According to some embodiments, optical sensordevice 120 may exhibit a photocurrent for over 5 hours after exposure toelectromagnetic radiation has stopped.

Image capture device 110 may also comprise an exposure control device130, to control the exposure of optical sensor device 120 to light or toparticular wavelengths of electromagnetic radiation. According to someembodiments, exposure control device 130 may comprise at least one of ashutter, filter or lens. According to some embodiments, exposure controldevice 130 may comprise one or more light sources. For example, exposurecontrol device 130 may comprise a first light source producing lighthaving a first wavelength that increases the conductance of or otherwiseproduces a write effect on optical sensor device 120, and a second lightsource producing light having a second wavelength that decreases theconductance of or otherwise produces an erase effect on optical sensordevice 120. Exposure control device 130 may be located proximate tooptical sensor device 120.

Image capture device 110 further comprises a processor 140, which maycomprise one or more microprocessors, central processing units (CPUs),application specific instruction set processors (ASIPs), or otherprocessor capable of reading and executing instruction code. Accordingto some embodiments, processor 140 may be located on a single chip withoptical sensor device 120. According to some embodiments, processor 140may also be located on a single chip with long-term memory 150.Processor 140 may be configured to control the operation of exposurecontrol device 130 to write and erase data from optical sensor device120. Processor 140 may also be configured to read data from opticalsensor device 120. According to some embodiments, reading data fromoptical sensor device 120 may comprise applying a voltage to opticalsensor device 120 and determining a conductance or resistance of thedevice based on the applied voltage.

Image capture device 110 may further comprise long-term memory 150storing program code 151 and data 159. Processor 140 may be configuredto access long-term memory 150, to execute instructions stored inprogram code 151 and to read from and write to data 159. According tosome embodiments, processor 140 may be configured to read data fromoptical sensor device 120 and write the data to data 159 of long-termmemory 150 for long term storage. Long-term memory 150 may comprise oneor more volatile or non-volatile memory types, such as RAM, ROM, EEPROM,or flash, for example.

Program code 151 of long-term memory 150 may comprise a plurality ofcode modules executable by processor 140 to cause image capture device110 to capture, store and process image data. For example, program code151 may include a data capture module 152, an exposure control module153, a handwriting recognition module 154 and a facial recognitionmodule 155.

When processor 140 executes data capture module 152, processor 140 maybe caused to apply a voltage to optical sensor device 120, determine atleast one conductance of optical sensor device 120, and to write data todata 159 based on the at least one conductance.

When processor 140 executes exposure control module 153, processor 140may be caused to control the functions of exposure control device 130.Where exposure control device 130 comprises a shutter, the control mayinclude opening and/or closing the shutter, for example. Where exposurecontrol device 130 comprises a light source, the control may includeturning the light source on and off.

When processor 140 executes handwriting recognition module 154,processor 140 may be caused to control cause optical sensor device 120to act as a neural network, and to train optical sensor device 120 torecognize images of handwritten characters, as described below infurther detail with reference to FIGS. 8 and 9 .

When processor 140 executes facial recognition module 155, processor 140may be caused to control cause optical sensor device 120 to act as aneural network, and to train optical sensor device 120 to recognizeimages of faces, as described below in further detail with reference toFIGS. 10 and 11 .

Image capture device 110 further includes user I/O 160, which mayinclude one or more forms of user input and/or output devices, such asone or more of a screen, keyboard, mouse, touch screen, microphone,speaker, or other device that allows information to be delivered to orreceived from a user. Image capture device 110 may also includecommunications module 170. Communications module 170 may be configuredto communicate with one or more external computing devices or computingsystems, via a wired or wireless communication protocol. For example,communications module 170 may facilitate communication via at least oneof Wi-Fi, Bluetooth, Ethernet, USB, or via a cellular network in someembodiments. In the illustrated embodiment, communications module 170 isin communication with a remote device 180, which may be a computer insome embodiments.

FIG. 2 shows a detailed view of optical sensor device 120 with exposurecontrol device 130. In the illustrated embodiments, exposure controldevice 130 comprises two light sources 132 and 134, with light source132 producing light with a 280 nm wavelength and light source 134producing light with a 365 nm wavelength. However, as described above,exposure control device 130 may comprise a different mechanism forselectively exposing optical sensor device 120 to light or otherelectromagnetic radiation. For example, exposure control device 130 maycomprise a first filter for filtering electromagnetic radiation of afirst wavelength, a second filter for filtering electromagneticradiation of a second wavelength and a shutter for blocking allelectromagnetic radiation. Processor 140 manipulating exposure controldevice 130 may therefore be able to cause only electromagnetic radiationof the first wavelength, only electromagnetic radiation of the secondwavelength, or no electromagnetic radiation to be received by opticalsensor device 120.

Optical sensor device 120 may comprise an optoelectronic memory devicefabricated in a phototransistor configuration as described inInternational Application No. PCT/AU2019/050662. In the embodimentillustrated in FIG. 2 , optical sensor device 120 comprises a sensingelement 121, which may be a black phosphorous layer comprising multiplevertically stacked layers of black phosphorous. According to someembodiments, sensing element 121 may be comprise an atomically thinsemiconductor material. According to some embodiments, the thickness ofthe semiconductor material may be in the range of 2 nm to 50 nm.According to some embodiments, this may be produced by a process ofmechanical exfoliation and dry transfer.

By exploiting oxidation induced defects on the layers of blackphosphorous, light tenability can be achieved, causing the layers togenerate a unique persistent photo-response under the illumination ofparticular wavelengths of light or electromagnetic radiation. Asdescribed above, the conductance exhibited by the black phosphorouslayers may increase when exposed to light with a wavelength in the rangeof 250-315 nm, and may decrease when exposed to light with a wavelengthin the range of 315-400 nm. This effect can allow for multi-bitprogramming and erasing functions, image pre-processing and neuromorphicimage recognition functions to be performed. According to someembodiments, while black phosphorous may exhibit an anisotropicphoto-response, the above-described wavelength specific increases anddecreases in conductance may be exhibited regardless of the crystalorientation of the black phosphorous.

According to some embodiments, optical sensor device 120 may furthercomprise a passivation layer 122, which may be a phosphorous oxide(P_(x)O_(y)) layer in some embodiments. The native phosphorous oxidelayer may form on top of black phosphorous substrate 121 due to theexposure of substrate 121 with oxygen, and may act as a self-passivationlayer for the black phosphorous to protect it from further corrosion.Furthermore, layer 122 may induce localized trap sites for chargecarriers, causing unusual negative photoconductivity under opticalexcitation, allowing for the unique band selective photo-response underdifferent wavelengths of electromagnetic radiation as described above.According to some embodiments, passivation layer 122 may form on boththe top and bottom of black phosphorous substrate 121. According to someembodiments, layer 122 may be between 1 and 5 nm thick.

Optical sensor device 120 further comprises an electrically insulatingsubstrate 123, which may be a SiO₂/Si layer 123.

Optical sensor device 120 further comprises electrodes 124, which allowa voltage to be applied between them and to pass through layer 121.According to some embodiments, the electrodes may each comprise twolayers, such as a chromium layer and a gold layer, for example.

Layers 121, 122 and 123, and electrodes 124 may be positioned on a backgate 125. Back gate 125 may be configured to act as a third electrode,and may allow optical sensor device 120 to perform gate-assisted writeand erase functions, as described in further detail below with referenceto FIGS. 6A and 6B.

FIG. 13 shows a method of capturing image data using image capturedevice 110.

At step 1310, processor 140 executing exposure control module 153 iscaused to control exposure control device 130 in order to cause exposurecontrol device 130 to adopt a exposure mode, in which exposure controldevice 130 causes optical sensor device 120 to be exposed to firstelectromagnetic radiation. Where exposure control device 130 comprises ashutter or filter, adopting the exposure mode may comprise opening theshutter or filter. Where exposure control device 130 comprises a sourceof electromagnetic radiation such as a light 132, adopting the exposuremode may comprise switching on the source of electromagnetic radiation.The electromagnetic radiation may be of a first predeterminedwavelength, which may cause an increase in conductance of optical sensordevice 120 or otherwise cause data to be written to optical sensordevice 120.

At step 1320, processor 140 executing exposure control module 153 iscaused to control exposure control device 130 in order to cause exposurecontrol device 130 to adopt a protected mode, in which optical sensordevice 120 is blocked from or otherwise no longer exposed to the firstelectromagnetic radiation. According to some embodiments, processor 140may execute step 1320 after a predetermined period of time has elapsedafter performing step 1310. Where exposure control device 130 comprisesa shutter or filter, adopting the protected mode may comprise closingthe shutter or filter. Where exposure control device 130 comprises asource of electromagnetic radiation such as a light 132, adopting theprotected mode may comprise switching off the source of electromagneticradiation.

At step 1330, processor 140 executing data capture module 152 is causedto read reading data captured by optical sensor device 120. According tosome embodiments, processor 140 may be caused to apply a voltage tooptical sensor device 120 in order to determine a conductance orresistance of the optical sensor device 120 based on the appliedvoltage. According to some embodiments, the measured conductance orresistance may be compared with one or more predetermined thresholdvalues to determine a value stored by optical sensor device 120.

At step 1340, processor 140 executing data capture module 152 may becaused to store the value read from optical sensor device to data 159 oflong term memory 150.

In some embodiments, processor 140 may then be configured to performsteps 1350 and 1360 to erase the data from optical sensor device 120.

At step 1350, processor 140 executing exposure control module 153 iscaused to control exposure control device 130 in order to cause exposurecontrol device 130 to adopt a exposure mode, in which exposure controldevice 130 causes optical sensor device 120 to be exposed to secondelectromagnetic radiation. Where exposure control device 130 comprises ashutter or filter, adopting the exposure mode may comprise opening theshutter or filter. Where exposure control device 130 comprises a sourceof electromagnetic radiation such as a light 132, adopting the exposuremode may comprise switching on the source of electromagnetic radiation.The electromagnetic radiation may be of a second predeterminedwavelength, which may cause a decrease in conductance of optical sensordevice 120 or otherwise cause data to be erased from optical sensordevice 120.

At step 1360, processor 140 executing exposure control module 153 iscaused to control exposure control device 130 in order to cause exposurecontrol device 130 to adopt a protected mode, in which optical sensordevice 120 is blocked from or otherwise no longer exposed to the secondelectromagnetic radiation. According to some embodiments, processor 140may execute step 1320 after a predetermined period of time has elapsedafter performing step 1310. Where exposure control device 130 comprisesa shutter or filter, adopting the protected mode may comprise closingthe shutter or filter. Where exposure control device 130 comprises asource of electromagnetic radiation such as a light 132, adopting theprotected mode may comprise switching off the source of electromagneticradiation.

FIGS. 3A and 3B show graphs illustrating the optical modulation andmemory function of an optical sensor device 120 under two illuminationwavelengths. FIG. 3A relates to a scenario in which sensor device 120was exposed to a 280 nm light source such as light source 132, whileFIG. 3B relates to a scenario in which sensor device 120 was exposed toa 365 nm light source such as light source 134. In each case, a voltageof 50 mV was applied to sensor device 120 via electrodes 124 (thedrain-source voltage V_(DS)), while the voltage at back gate 125 (thegate-source voltage V_(GS)) was held at 0 V.

Turning to FIG. 3A, the illustrated graph 300 has an X-axis 310displaying time in seconds, and a Y-axis 320 displaying the normalizedcurrent I_(DS). Bars 330 correspond to times at which sensor device 120was exposed to the 280 nm light source 132, and line 340 illustrates thecurrent measured in sensor device 120 at each point in time,corresponding to the normalized transient photo-response of sensordevice 120 to the illumination pulses illustrated by bars 330. As sensordevice 120 is exposed to the light, a positive photocurrent is generatedresulting in an increase in the measured current. This is due to thetrapping of photo-excited charge carriers at localized trap sitesinduced by the surface oxidation of the sensing element 121, asdescribed above with reference to FIG. 2 . When the exposure is stopped,the current gradually decreases over time, due to the release of thephoto-excited charge carriers.

Turning to FIG. 3B, the illustrated graph 350 has an X-axis 360displaying time in seconds, and a Y-axis 370 displaying the normalizedcurrent I_(DS). Bars 380 correspond to times at which sensor device 120was exposed to the 365 nm light source 134, and line 390 illustrates thecurrent measured in sensor device 120 at each point in time,corresponding to the normalized transient photo-response of sensordevice 120 to the illumination pulses illustrated by bars 380. As sensordevice 120 is exposed to the light, a negative photocurrent is generatedresulting in a reduction to the measured current. This is due to therelease of photo-excited charge carriers at localized trap sites inducedby the surface oxidation of the sensing element 121, as described abovewith reference to FIG. 2 . When the exposure is stopped, the currentgradually increases over time, due to the trapping of furtherphoto-excited charge carriers.

FIGS. 4A, 4B and 4C show graphs illustrating the dynamic memory accessoperation of optical sensor device 120 in an all-optical memory mode,taking advantage of the phenomenon illustrated by FIGS. 3A and 3B. Inthe all-optical memory mode, both write and erase functions can beachieved by exposing optical sensor device 120 to electromagneticradiation in the form of optical pulses, which may be done by processor140 controlling exposure control device 130. Read functions areperformed by applying voltage pulses to electrodes 124, but no voltageis applied to gate 125.

Turning to FIG. 4A, the illustrated graph 400 has an X-axis 405displaying time in milliseconds, and a Y-axis 410 displaying the powerin mW/cm² of illumination supplied to optical sensor device 120 bylights 132 and 134. Bars 415 correspond to times at which sensor device120 was exposed to the 280 nm light source 132, and bars 420 correspondto times at which sensor device 120 was exposed to the 365 nm lightsource 134.

FIG. 4B shows a graph 430 having an X-axis 435 displaying time inmilliseconds, and a Y-axis 440 displaying the voltage in mV applied toelectrodes 124 for the purpose of reading data from optical sensordevice 120. Bars 445 correspond to times at which the voltage pulseswere applied, and line 450 corresponds to the voltage applied over time,being 50 mV during a read pulse and 0V between pulses.

FIG. 4C shows a graph 460 having an X-axis 465 displaying time inmilliseconds, and a Y-axis 470 displaying the normalized current I_(DS)produced by optical sensor device 120. Bars 475 correspond to times atwhich the voltage pulses were applied. Bars 480 correspond to times atwhich sensor device 120 was exposed to 280 nm light source 132, and bars485 correspond to times at which sensor device 120 was exposed to 365 nmlight source 134. Line 490 illustrates the current measured in sensordevice 120 at each point in time, corresponding to the normalizedtransient photo-response of sensor device 120 to the illumination pulsesillustrated by bars 480 and 485. When sensor device 120 is exposed tothe 280 nm light, a positive photo-current is generated resulting in anincrease in the measured current. When sensor device 120 is exposed tothe 365 nm light, a negative photo-current is generated resulting in adecrease in the measured current.

In reference to FIGS. 4A, 4B and 4C, it is shown that initially at 0milliseconds, a 40 ms “read” voltage pulse of 50 mV is applied toquantify the “off” state of the current of optical sensor device 120with no exposure to light. As shown, the current is measured to bearound 0.33 μA. To program optical sensor device 120, at 40 ms opticalsensor device 120 is exposed to a “write” light pulse of 280 nm with apower density of 3.5 mW/cm² for 10 ms, while no voltage is applied.Subsequently, another “read” voltage pulse is applied to measure the now“on” state current of optical sensor device 120. As the “write” pulsehas generated photo-excited carriers, the current shows an increase,being at around 0.61 μA. During the “read” pulse, the current slowlydecays over time.

The relatively slow recovery of the current after exposure to the lightsources can be attributed to the charge carrier recombination throughoxidation induced in localized trap centers within the optical sensordevice 120, as described above with reference to FIG. 2 . As such,optical sensor device 120 shows a persistent photo-current even afterillumination is removed, and this residue photo-current persists evenafter a long retention time. According to some embodiments, opticalsensor device 120 may retain some photo-current for over 5.5 hours.

To reset optical sensor device 120, at 90 ms optical sensor device 120is exposed to an “erase” light pulse of 365 nm with a power density of7.3 mW/cm² for 10 ms, while no voltage is applied. A comparativelyhigher power density “erase” pulse may be used compared to the “write”pulse to compensate for the magnitude difference in photo-current.Subsequently, another “read” voltage pulse is applied to measure the now“off” state current of optical sensor device 120. As the “erase” pulsehas induces a negative photo-current, the current shows a decrease toits initial “off” state of around 0.33 μA.

Further “write”, “read” and “erase” actions are performed asillustrated. According to some embodiments, optical sensor device 120may be capable of performing at least 2000 “write” and “erase” cycleswithout any deterioration in cyclic endurance or stability, indicatingrepeatability and reproducibility of such a device.

As described above, optical sensor device 120 may be able to deliverultrafast operation speeds, as it is triggered with photonic signals.This allows sensor device 120 to be used in applications such asin-pixel image processing and facial recognition to realize efficienthardware for artificial intelligence, as described in further detailbelow. In contrast, previous optoelectronic memories have showncomparatively lower switching speeds (in milliseconds to seconds) thanknown electronic memories (which have switching speeds in the range ofnanoseconds to sub-microseconds). This is because in electronicmemories, fast charge injection through localized switching paths orfloating gate charge storage render high switching speed. On the otherhand, material dependent dynamics of photo-excited charge carries andassociated mechanisms such as trapping/de-trapping can be responsiblefor the slow operational speeds in previous optoelectronic memories.

In contrast, optical sensor device 120 may exhibit switching speed withoptical pulses of around 10 ms duration, being more than two orders ofmagnitude higher than some previous known optoelectronic memories.

Due to the persistent photo-current, slow recovery and long retentiontime, optical sensor device 120 may be used for multi-bit dataprogramming and storage. FIGS. 5A, 5B and 5C shows graphs illustratingoptical sensor device 120 being used for writing, erasing and reading8-bit data.

FIG. 5A shows a graph 500 having an X-axis 505 displaying time inseconds, and a Y-axis 510 displaying the power in mW/cm² of illuminationsupplied to optical sensor device 120 by a light source that induces an“erase” action. For example, the illumination may be a 365 nm lightpulse emitted by a light source such as light source 134. Line 515 showsthe power of illumination supplied to optical sensor device 120 by light134 over time, showing a single 15 second pulse of 8.5 mW/cm² ofillumination at around 460 seconds.

FIG. 5B shows a graph 530 having an X-axis 535 displaying time inseconds, and a Y-axis 540 displaying the power in mW/cm² of illuminationsupplied to optical sensor device 120 by a light source that induces a“write” action. For example, the illumination may be a 280 nm lightpulse emitted by a light source such as light source 132. Line 545 showsthe power of illumination supplied to optical sensor device 120 by light132 over time, showing a seven 5 second pulse of 3.5 mW/cm² ofillumination with a period of 55 seconds.

FIG. 5C shows a graph 560 having an X-axis 565 displaying time inseconds, and a Y-axis 570 displaying the current in μA produced byoptical sensor device 120 when exposed to the pulses of FIGS. 5A and 5B,as well as a voltage of 50 mV applied to electrodes 124. As shown, themultiple “write” pulses cause the current generated by sensor device 120to increase in steps, until the “erase” pulse causes the current to dropbelow the reference current or “off” current exhibited by sensor device120 when it is not exposed to any light sources. The current thenrecovers back to the reference or “off” current after termination of the“erase” pulse. This demonstrates optical sensor device 120 being able toadopt at least eight different memory states, which can be programmedand erased electronically. This allows optical memory device 120 to beused for optoelectronic in-memory computation and logic operations suchas image processing and neuromorphic computation, as described infurther detail below.

As well as being able to store and erase data purely optically, opticalsensor device 120 may further be able to be used in a gate-assistedmode, where voltage can be applied to back gate 125 to perform furtherprocessing functions. In this case, optical sensor device 120 mayoperate in a three-terminal field effect transistor configuration, whereillumination applied to optical sensor device 120 may be used tomodulate the transfer characteristics.

FIGS. 6A and 6B show example transfer characteristics of optical sensordevice 120 acting in an optical mode, and a gate-assisted mode.

FIG. 6A shows a graph 600 showing example transfer characteristics ofoptical sensor device 120 acting in an optical-only mode, in which thevoltage V_(GS) at back gate 125 is 0V. X-axis 610 shows the voltageV_(DS) applied to electrodes 124 in volts, while Y-axis 620 shows theoutput drain-source current profiles IDS exhibited by optical sensordevice 120 in μA. Reference line 630 shows the current produced whenoptical sensor device 120 is not exposed to any illumination.

Lines 632 show the current produced by optical sensor device 120 underdifferent power densities of 280 nm illumination. As shown in graph 600,under 280 nm illumination, I_(DS) increases with respect to referenceline 630 as the power density increases. Lines 634 show the currentproduced by optical sensor device 120 under different power densities of365 nm illumination. As shown in graph 600, under 365 nm illuminationIDS decreases proportionally with respect to reference line 630 as thepower density increases.

FIG. 6B shows a graph 650 showing example transfer characteristics ofoptical sensor device 120 acting in a gate-assisted mode, in which thevoltage V_(GS) at back gate 125 is varied. X-axis 660 shows the gatevoltage V_(GS) applied to back gate 125, while Y-axis 670 shows theoutput drain-source current profiles IDS exhibited by optical sensordevice 120 in μA. Reference line 680 shows the current produced whenoptical sensor device 120 is not exposed to any illumination.

Lines 682 show the current produced by optical sensor device 120 underdifferent power densities of 280 nm illumination. Lines 684 show thecurrent produced by optical sensor device 120 under different powerdensities of 365 nm illumination. The calculated carrier mobilityincreases and decreases after illumination with 280 nm and 365 nmwavelengths, respectively, compared to reference line 680. This increasein mobility under 280 nm illumination may be associated with the highenergy of hot carriers gained under high energy excitation, while themobility decreases under 365 nm illumination may be due to the carrierscattering by charged defects under low energy excitation, as describedin further detail below. As such, the change in conductance of opticalsensor device 120 regulated by optical signals under the influence ofvoltage applied to back gate 125 allows for programming and erasingcapabilities by simply applying specified wavelengths, such as 280 nmand 365 nm wavelengths, respectively.

The transfer characteristics of optical sensor device 120 ingate-assisted mode demonstrate the power-dependent multi-level dataprogramming and erasing operations that can be achieved with opticalsensor device 120.

Further observing FIG. 6B, it can be clearly observed that the thresholdvoltages V_(TH) of the transfer curves shifts during gate-assisted“write” and “erase” operations. This V_(TH) shift is possibly due to theelectrostatic screening by light induced charge trapping at the surfaceof layers 121 and 122 of or bottom of the layer 121/122/123 interface.As shown in graph 650, during the “write” operation, V_(TH) steadilyincreases from +3.45 V to +27.35 V when the illumination power densityof the 280 nm illumination pulses increases from 0.5 mW/cm2 to 3 mW/cm2.During the “erase” operation, V_(TH) shifts from −4.8 V to −11.59 V asthe power density of the 365 nm illumination pulses increase from 2mW/cm2 to 10 mW/cm2. As such, in the gate-assisted configuration, amemory window is defined by the difference between the programming or“write” state V_(TH) and the “erase” state V_(TH). A maximum memorywindow of ˜38.94 V may be observed for power densities of 3 mW/cm2 and10 mW/cm2 during “write” and “erase” operations, for example.

According to some embodiments, optical sensor device 120 may show goodendurance and cyclic repeatability of the optical “write” and “erase”operations in gate-assisted mode. For example, in some embodiments,optical sensor device 120 may exhibit no significant degradation in thememory window after more than 500 consecutive “write” and “erase” cyclesin ambient conditions.

The described features, characteristics and properties of optical sensordevice 120, image capture device 110 and system 100 may allow them to beconfigured to perform image processing functions such as image detectionand memorization. Specifically, the deep trap-states assisted slowrecovery and persistent photo-current exhibited by optical sensor device120 allows for on-chip image-processing functions such as weak signalaccumulation for real-time image enhancement. Furthermore, in-pixelstorage can be used to achieve massive parallel computation.

According to some human memory models, external information received byhuman sensory organs is stored as a sensory memory for a very short timeand then selected information is transferred from a temporary short-termmemory (STM) to a permanent long-term memory (LTM). The STM correspondsto temporally weak neural plasticity which persists for a shortduration, ranging from a few seconds to minutes. However, via theprocess of consolidation which involves repetitive stimuli and frequentrehearsals, the STM transforms into LTM, which corresponds to atemporally stronger neural plasticity lasting from several minutes toyears. Using optical pulses as stimuli for optical sensor device 120 andvarying their frequency by controlling exposure control device 130,these human memory behaviors including a transition of STM to LTM andoptically tunable synaptic plasticity can be imitated by image capturedevice 110.

FIGS. 7A and 7B show examples of optical sensor device 120 exhibitingSTM and LTM.

FIG. 7A shows a diagram 700 in which an optical sensor device 120 iscaused to exhibit STM. The illustrated optical sensor device 120 is a2×2 pixel chip, comprising four pixels I, II, III and IV. Each pixelcomprises layers 121, 122 and 123 as well as electrodes 124, all sharinga single back gate 125. Exposure control device 130, which may be twolight sources 132 in some embodiments, is caused to illuminate twopixels with a series of “write” illumination pulses, which may be 280 nmpulses in some embodiments. In the illustrated embodiment, pixels II andIV are illuminated with 100 consecutive short optical pulses with pulsewidth of 50 ms and a power density of 3 mW/cm2 at 1 Hz.

Images 720, 730 and 740 shows the current of each pixel 710 measured atvarious times. For comparison, the current of each pixel is scaledbetween 0 and 1 as shown in key 701, with 0 corresponding to the minimummeasured current, being around 0.95 μA, and 1 corresponding to themaximum measured current, being around 3.93 μA.

Image 720 shows the current of each pixel 710 measured before theillumination pulses are delivered. The current of each pixel 710 isrelatively low.

Image 730 shows the current of each pixel 710 measured within 0.5seconds of the illumination pulses being delivered. Pixels II and IVshow a relatively high current compares to pixels I and III,corresponding to the illumination pulses they have received.

Image 740 shows the current of each pixel 710 measured 10 minutes afterthe illumination pulses delivered. The current of each pixel 710 hasreturned to a similar level as that exhibited by pixels 710 beforereceiving the illumination pulses. This corresponds to a STM effect.

FIG. 7B shows a diagram 750 in which an optical sensor device 120 iscaused to exhibit LTM. The illustrated optical sensor device 120 is thesame 2×2 pixel chip shown in FIG. 7A. Exposure control device 130, whichmay be two light sources 132 in some embodiments, is caused toilluminate two pixels with a series of “write” illumination pulses,which may be 280 nm pulses in some embodiments. In the illustratedembodiment, pixels I and III are illuminated with 100 consecutive shortoptical pulses with pulse width of 50 ms and a power density of 3mW/cm2, this time at 10 Hz.

Images 770, 780 and 790 shows the current of each pixel 710 measured atvarious times. For comparison, the current of each pixel is scaledbetween 0 and 1 as shown in key 701, being the same scale used in FIG.7A.

Image 770 shows the current of each pixel 710 measured before theillumination pulses are delivered, and after an erase has beenperformed. The current of each pixel 710 is relatively low.

Image 780 shows the current of each pixel 710 measured within 0.5seconds of the illumination pulses being delivered. Pixels I and IIIshow a relatively high current compares to pixels II and IV,corresponding to the illumination pulses they have received. Thedifference in current is even greater than that exhibited by the pixelsas shown in image 730 of FIG. 7A.

Image 790 shows the current of each pixel 710 measured 10 minutes afterthe illumination pulses delivered. In this case, while the current ofeach pixel 710 has started returning to a similar level as thatexhibited by pixels 710 before receiving the illumination pulses, thedifference between the illuminated pixels I and III and theunilluminated pixels II and IV is still measurable. This corresponds toa LTM effect. The increase in conductance contrast within optical sensordevice 120 at higher illumination frequencies highlights the capabilityof using image capture device 110 for in-pixel image enhancement andreal-time data processing of the input visual information.

The described features, characteristics and properties of optical sensordevice 120, image capture device 110 and system 100 may further allowthem to be configured to perform machine learning functions. Forexample, FIG. 8 shows a method of training image capture device 110 toact as an optical neural network to perform handwriting recognition. Inthe illustrated embodiments, optical sensor device 120 acts as a neuralnetwork comprising an input layer 820, a weight matrix 830 and an outputlayer 840. In the illustrated embodiments, input layer 820 comprises28×28 input neurons numbered X₀ to X₇₈₃, and output layer 840 comprises10 output neurons numbered Y₀ to Y₉. Each input neuron may comprise apixel 710. The input neurons are connected to the output neurons throughweight matrix 830, which in the illustrated embodiment comprises 784×10synaptic weights, with each input neuron being connected to each outputneuron through an individual synaptic weight. Each synaptic weight isthe conductance of the pixel 710, and depends on intensity or dosage ofthe light exposed onto the pixel 710.

Optical sensor device 120 is trained via a single-layer perceptron modelperforming supervised learning with a back-propagation algorithm, usinga plurality of training images. For example, FIG. 8 shows an example28×28 pixel input image 810, being a handwritten “0” digit. Eachtraining image may be associated with a label value k_(n). For example,the label k_(n) associated with input image 810 may be the value “0”.The input images may be obtained from a database such as the ModifiedNational Institute of Standards and Technology (MNIST) dataset.According to some embodiments, between 10,000 and 100,000 trainingimages may be used.

During training, each neuron in the input layer 820 receives stimulationcorresponding to a pixel in the image 810 and is assigned to an inputvector X_(m). Input vector X_(m) is then transformed to 10 output valuesΣ_(n) of output layer 840 through weigh matrix 830 (W_(m,n)) to feed theoutput neurons Y₀ to Y₉. The values of Σ_(n) are converted to an outputvector Y_(n) by a sigmoid activation function. For example, outputvector Y_(n) may be determined by calculating:

$Y_{n} = {{f\left( {\sum}_{n} \right)} = \frac{1}{1 + e^{- {\sum}_{n}}}}$

Next, the difference between the values of the output vector Y_(n) andthe label value k_(n) associated with the input image is used to updatethe synaptic weights of weight matric 830 through a weight update methodbased on the backpropagation algorithm. Specifically, the sign of theweight change sgn(ΔW) is calculated using the difference between theoutput value Y_(n) and each label value k_(n) to determine whether thesynaptic weight needs to increase or decrease, such that:

sgn(ΔW)>0 if k _(n) Y _(n)>0

sgn(ΔW)<0 if k _(n) Y _(n)<0

Where sgn(ΔW)>0, both the synaptic weight W and the device conductance Gare increased. Conductance G refers to the real conductance value ofoptical sensor device 120, while synaptic weight W refers to theconductance value calculated by the neural network during recognition.Where sgn(ΔW)<0, both the synaptic weight W and the device conductance Gare decreased.

During training, the calculation of synaptic weight includes bothpositive and negative values, while the conductance of optical sensordevice 120 is always positive. To overcome this discrepancy, thesynaptic weight W may be considered to be the difference of twoconductance values, being G_(m,n) and G_(int), whereG_(int)=(G_(max)+G_(min))/2. G_(max) and G_(min) may be calculated basedon experimental data. According to some embodiments, only G_(m,n) may beused to modify the synaptic weights W of the network, while Gint may beused purely as a scaling factor for normalization.

Once optical sensor device 120 is trained using the method describedabove, it can be used to label input images that were not part of thetraining dataset.

In one example, an optical sensor device 120 was trained using 60,000images from the MNIST dataset. Sensor device 120 was then asked to label10,000 test images which were not provided during network training, andthe recognition accuracy was determined. FIGS. 9A and 9B shows mappingimages of a representative input handwritten digit ‘0’ before and after25,000 training phases. FIG. 9A shows a heat map 900 showing theclassification accuracy in an initial state before training, while FIG.9B shows a heat map 950 showing classification accuracy in a final stateafter training. Key 901 shows how the areas of the heat maps correspondto classification accuracy as a percentage between 0% and 100%.

The recognition accuracy was calculated for different pulse widths (1,20, 50, 100 and 300 ms) of optical stimuli applied to optical sensordevice 120, and a maximum recognition accuracy of ˜90% was achieved.

A similar process can be used to train image capture device 110 to actas an optical neural network to perform face recognition, as shown inFIG. 10 . In the illustrated embodiments, optical sensor device 120 actsas a neural network comprising an input layer 1020, a weight matrix 1030and an output layer 1040. In the illustrated embodiments, input layer1020 comprises 60×60 input neurons numbered I₀ to I₃₅₉₉, and outputlayer 1040 comprises a single output neuron Z_(out). The input neuronsare connected to the output neurons through weight matrix 1030, which inthe illustrated embodiment comprises 3600×1 synaptic weights, with eachinput neuron being connected to the output neuron through an individualsynaptic weight.

Optical sensor device 120 is trained via a single-layer perceptron modelperforming supervised learning with a back-propagation algorithm, usinga plurality of training images. For example, FIG. 10 shows an example60×60 greyscale pixel input image 1010 of a face.

During training, the weight matrix 1030 is updated based on theexperimental conductance modification of optical sensor device 120depending on whether an increase or decrease in the synaptic weight isrequired, as described above with reference to FIG. 8 . The highest andlowest synaptic weights correspond to white and black pixels of thegrayscale input image. In each learning epoch, 150 pixels are randomlyselected to update the synaptic weight.

FIG. 11 shows a series of images 1100 showing the evolution of the imageduring the learning process at various stages. As depicted by FIG. 11 ,when optical sensor device 120 is trained with stimulation pulses of 20milliseconds, sensor device 120 exhibits a relatively slower learningrate but a higher accuracy, as the facial details in the output imageafter 800 epochs are better at the final state than in the otherdatasets.

In contrast, training with the highest nonlinearity dataset using 300millisecond pulses gives a faster learning rate, as the facial contourappear after relatively lesser training epochs, but gives the lowestaccuracy with a less clear image at the final state. Using a StructuralSimilarity method to measure the difference between the learnt image andthe input image, it was found that the final recognition accuracy of aminimal nonlinearity system is higher, at around 96% in 837 epochs for 1millisecond pulses, than that of maximum nonlinearity system, which gavean accuracy of only 53.67% in 204 epochs for 300 millisecond pulses.This result can be associated with the higher rate of weight change in anonlinearity system and the almost even distribution of synaptic weightsin a minimal nonlinearity system. These results show that the inherentoptical information detection, in-pixel processing and retentioncharacteristics of the optical sensor device 120 make it possible toconfigure sensor device 120 to successfully imitate human visual memory.

FIG. 1200 shows a flowchart illustrating a method 1200 of trainingoptical sensor device 120 to act as a neural network for performingimage recognition. Method 1200 may be performed by processor 140executing program code 130.

At step 1210, processor 140 executing handwriting recognition module 154or facial recognition module 155 receives training data from remotedevice 180 via communications module 170. The training data may comprisetraining images or handwritten text, faces, or other images.

At step 1220, processor 140 executes exposure control module 153 tocause exposure control device 130 to expose optical sensor device 120 toa first image from the received training data for a predetermined periodof time. This may be by causing one or more light sources correspondingto pixels of the image to illuminate one or more pixels 710 of opticalsensor device 120.

At step 1230, processor 140 executing handwriting recognition module 154or facial recognition module 155 determines an output value by applyinga weight matrix stored in memory 150 to the output exhibited by sensordevice 120 in response to the test image.

At step 1240, processor 140 executing handwriting recognition module 154or facial recognition module 155 compares the output value with a labelassociated with the test image being processed.

At step 1250, processor 140 adjusts the weight matrix stored in memory150 based on the outcome of the comparison performed at step 1240.

At step 1260, processor 140 determines whether further training isrequired. This may be by comparing the number of times that opticalsensor device 120 has been exposed to the current test image, and bycomparing the number of test images already used with the total numberof training images available.

If processor 140 determines that further training is required, then atstep 1270 processor 140 executes exposure control module 153 to againcause exposure control device 130 to expose optical sensor device 120 toan image for a predetermined period of time. The image may be the sameas that already exposed to optical sensor device, or may be a new image.Processor 140 then proceeds to perform the method from step 1230.

If processor 140 determines that no further training is required, thenat step 1280 processor 140 stores the final weight matrix 1280 in memory150. This can then be used to process future images for the purposes ofimage recognition.

It will be appreciated by persons skilled in the art that numerousvariations and/or modifications may be made to the above-describedembodiments, without departing from the broad general scope of thepresent disclosure. The present embodiments are, therefore, to beconsidered in all respects as illustrative and not restrictive.

What is claimed is:
 1. An imaging device for capturing image data, theimaging device comprising: an optical sensor responsive toelectromagnetic radiation, wherein the optical sensor exhibits aphoto-memory effect causing the optical sensor to act as a short-termmemory device; an exposure control device configured to adopt a firstmode and a second mode, wherein in the first mode the exposure controldevice causes electromagnetic radiation to reach the optical sensor andin the second mode the exposure control device restricts electromagneticradiation from reaching the optical sensor; a processor configured toreceive sensor data from the optical sensor, and to cause the exposurecontrol device to selectively adopt the first mode and the second mode;and a long term memory device accessible to the processor, wherein theprocessor is configured to read executable instructions from the longterm memory device and to write sensor data to the long term memorydevice, wherein data stored in the optical sensor acting as theshort-term memory device is erasable by exposing the optical sensor toelectromagnetic radiation.
 2. The imaging device of claim 1, wherein theoptical sensor is responsive to one or more of: ultraviolet light,visible light, and infrared light.
 3. The imaging device of claim 1,wherein the optical sensor comprises a two-dimensional semiconductor. 4.The imaging device of claim 3, wherein the two-dimensional semiconductorcomprises at least one of: an elemental material, transitional metaloxide, or transition metal chalcogenide.
 5. The imaging device of claim4, wherein the two-dimensional semiconductor comprises blackphosphorous.
 6. The imaging device of claim 1, wherein the opticalsensor comprises an array of optical sensor devices.
 7. The imagingdevice of claim 1, wherein the optical sensor is configured to exhibit areference conductance in an absence of electromagnetic radiation, and toexhibit a wavelength dependent increase and/or decrease in conductancewhen exposed to electromagnetic radiation.
 8. The imaging device ofclaim 7, wherein the conductance of the optical sensor is configured to:(i) return to the reference conductance not less than 1 second afterexposure to the electromagnetic radiation has ceased; or (ii) return tothe reference conductance not less than 1 minute after exposure to theelectromagnetic radiation has ceased.
 9. The imaging device of claim 7,wherein the optical sensor is configured such that the conductanceincreases when the optical sensor is exposed to electromagneticradiation of a first wavelength, and decreases when the optical sensoris exposed to electromagnetic radiation of a second wavelength.
 10. Theimaging device of claim 9, wherein the first wavelength is between 250and 315 nm, and the second wavelength is between 315 and 400 nm.
 11. Theimaging device of claim 7, wherein the processor is configured to readdata stored on the optical sensor by applying a voltage to the opticalsensor.
 12. The imaging device of claim 1, wherein the exposure controldevice comprises at least one of a shutter configured to blockelectromagnetic radiation or a filter configured to filter out at leastsome wavelengths of electromagnetic radiation.
 13. The imaging device ofclaim 1, wherein the exposure control device comprises at least onesource of electromagnetic radiation.
 14. A method of capturing imagedata, the method comprising: operating an exposure control device toadopt a first mode, wherein in the first mode the exposure controldevice causes an optical sensor responsive to electromagnetic radiationto be exposed to first electromagnetic radiation, wherein the opticalsensor exhibits a photo-memory effect causing the optical sensor to actas a short-term memory device; upon determining that a threshold periodof time has elapsed, operating the exposure control device to adopt asecond mode, wherein in the second mode the exposure control devicecauses the optical sensor to be blocked from electromagnetic radiation;reading data captured by the optical sensor based on the exposure to thefirst electromagnetic radiation; and recording the read data to a longterm memory device.
 15. The method of claim 14, wherein the method isperformed by an imaging device, the imaging device comprising: theoptical sensor responsive to electromagnetic radiation, wherein theoptical sensor exhibits the photo-memory effect causing the opticalsensor to act as the short-term memory device; the exposure controldevice; a processor configured to receive sensor data from the opticalsensor, and to cause the exposure control device to selectively adoptthe first mode and the second mode; and the long term memory device, thelong term memory device accessible to the processor, wherein theprocessor is configured to read executable instructions from the longterm memory device and to write sensor data to the long term memorydevice; and wherein data stored in the optical sensor acting as theshort-term memory device is erasable by exposing the optical sensor toelectromagnetic radiation.
 16. The method of claim 14, furthercomprising erasing the read data from the optical sensor acting as ashort term memory device by operating the exposure control device toadopt a third mode, wherein in the third mode the exposure controldevice causes the optical sensor to be exposed to second electromagneticradiation.
 17. The method of claim 14, wherein the exposure controldevice comprises a shutter or filter, wherein adopting the first modecomprises opening the shutter or filter, and wherein adopting the secondmode comprises closing the shutter or filter.
 18. The method of claim14, wherein the exposure control device comprises a source ofelectromagnetic radiation, wherein adopting the first mode comprisesswitching on the source of electromagnetic radiation, and whereinadopting the second mode comprises switching off the source ofelectromagnetic radiation.
 19. A method of training a neural network,the method comprising: receiving at least one training image and a labelrelated to the at least one training image; capturing image data by:operating an exposure control device to adopt a first mode, wherein inthe first mode the exposure control device causes an optical sensorresponsive to electromagnetic radiation to be exposed to firstelectromagnetic radiation, wherein: the optical sensor exhibits aphoto-memory effect causing the optical sensor to act as a short-termmemory device, and the first electromagnetic radiation is generatedbased on the at least one training image; and upon determining that athreshold period of time has elapsed, operating the exposure controldevice to adopt a second mode, wherein in the second mode the exposurecontrol device causes the optical sensor to be blocked fromelectromagnetic radiation; reading data captured by the optical sensorbased on the exposure to the first electromagnetic radiation; andrecording the read data to a long term memory device, thereby capturingimage data; determining an output based on the read data and a weightmatrix; comparing the read data with the label; and modifying the weightmatrix based on the comparison.
 20. The method of claim 19, furthercomprising repeating the method with: (i) the same at least one trainingimage; or (ii) a further training image different to the at least onetraining image.